.. _Using NeurEco with MATLAB: Tutorial: using NeurEco with MATLAB ========================================================= The following section will use the test case :std:ref:`Energy consumption test case`. This test case is delivered with the NeurEco installation package. .. note:: This was tested on Matlab 2017a and newer versions. A python 3 must be installed with a numpy package installed and the NeurEco package installed. .. note:: This tutorial is working with a **Regression** problem. The calls should be adapted when working with other solutions according to their Python API: * :std:ref:`Tabular Regression with the Python API` * :std:ref:`Tabular Compression with the Python API` * :std:ref:`Tabular Classification with the Python API` * :std:ref:`Discrete Dynamic with the Python API` The first thing to do is create a new directory and place the data files in it. Then change MATLAB working directory to the one just created. If multiple python environments are available, specify which version of python 3 will be used by running the following MATLAB command: .. code-block:: matlab pyversion(full path to the python executable of the environment to use) the next step is to load the data: .. code-block:: matlab x_test = dlmread('x_test.csv', ';', 1, 0); y_test = dlmread('y_test.csv', ';', 1, 0); x_train = dlmread('x_train.csv', ';', 1, 0); y_train = dlmread('y_train.csv', ';', 1, 0); Then proceed to create a regressor object: .. code-block:: matlab builder = py.NeurEco.NeurEcoTabular.Regressor(); The getattr python method can be used to access all the attributes of the model: .. code-block:: matlab version = char(py.getattr(builder, "__version__")); path = char(py.getattr(builder, "__path__")); methods = char(py.getattr(builder, "__methods__")); disp(strcat('version: ', version)) disp(strcat('path: ', path)) disp(strcat('methods: ', methods)) .. code-block:: text version:NeurEco Tabular version 4.01.2474.0 compiled with MSVC v1928 on Oct 12 2022 @ 17:09:04 path:Dynamic Library loaded from: C:\Program Files\Adagos\NeurEco\bin methods:**** NeurEco Tabular Regressor methods: **** - load - save - delete - evaluate - build - get_input_count - get_output_count - load_model_from_checkpoint - get_number_of_networks_from_checkpoint - get_weights - export_fmu - export_c - export_onnx - export_vba - compute_error - plot_network - forward_derivative - gradient - set_weights - perform_input_sweep To build the model, first choose the build settings. Note that the build method for a regressor takes only two required arguments, but in this case most of the optional arguments are set: .. code-block:: matlab write_model_to = './EnergyConsumption/EnergyConsumption.ednn'; checkpoint_address = './EnergyConsumption/EnergyConsumption.checkpoint'; inputs_shifting = 'min_centered'; inputs_scaling = 'max_centered'; outputs_shifting = 'auto'; outputs_scaling = 'auto'; inputs_normalize_per_feature = true; outputs_normalize_per_feature = false; valid_percentage = py.float(33.33); use_gpu = false; gpu_id = 0; checkpoint_to_start_build_from = ''; final_learning = true; disconnect_inputs_if_possible = true; initial_beta_reg = py.float(0.1); validation_output_data = py.None; validation_input_data = py.None; .. note:: When passing an argument, the Booleans and the chars are accepted as is. However, when it comes to numerical values the user has to specify the type: py.float, py.int... Same thing when it comes to None (py.None). More information is available about converting between MATLAB types and python types here: ``_ The next step is to convert the data from MATLAB doubles to numpy arrays: .. code-block:: matlab n_train_samples = py.int(size(x_train, 1)); n_inputs = py.int(size(x_train, 2)); n_outputs = py.int(size(y_train, 2)); input_train = py.numpy.reshape(py.numpy.array(reshape(x_train.',1,[])), py.tuple({n_train_samples, n_inputs})); output_train = py.numpy.reshape(py.numpy.array(reshape(y_train.',1,[])), py.tuple({n_train_samples, n_outputs})); The data is reshaped twice, because the conversion works only for 1-D arrays, so the 2D double arrays are flattened in MATLAB and reshaped back in python. Now that the data and the settings are ready, the build method can be called: .. code-block:: matlab builder.build(input_train, output_train, ... pyargs('write_model_to', write_model_to, ... 'checkpoint_address', checkpoint_address, ... 'inputs_shifting', inputs_shifting, ... 'outputs_shifting', outputs_shifting, ... 'inputs_scaling', inputs_scaling, ... 'outputs_scaling', outputs_scaling, ... 'outputs_normalize_per_feature', outputs_normalize_per_feature, ... 'inputs_normalize_per_feature', inputs_normalize_per_feature, ... 'valid_percentage', valid_percentage, ... 'checkpoint_to_start_build_from', checkpoint_to_start_build_from, ... 'final_learning', final_learning, ... 'use_gpu', use_gpu, ... 'gpu_id', gpu_id, ... 'initial_beta_reg', initial_beta_reg, ... 'disconnect_inputs_if_possible', disconnect_inputs_if_possible, ... 'validation_output_data', validation_output_data, ... 'validation_input_data', validation_input_data)... ); builder.delete(); .. note:: When passing the optional arguments, we need to use the pyargs method, but no need for that when the arguments are required. NeurEco will start building the model, and when it's done, the model will be saved in the directory ./EnergyConsumption. Intermediate models are saved to the checkpoint file, these models are accessible even before the end of the build. The following script show how to load them: .. code-block:: matlab model = py.NeurEco.NeurEcoTabular.Regressor(); n_models = double(model.get_number_of_networks_from_checkpoint('./EnergyConsumption/EnergyConsumption.checkpoint')); for i=1:n_models disp(strcat('Loading and evaluating model-', num2str(i), ' from checkpoint file.')) model.load_model_from_checkpoint('./EnergyConsumption/EnergyConsumption.checkpoint', py.int(i-1)); n_trainable_parameters = double(model.get_weights().size); disp(strcat('Number of trainable parameters for this temporary model: ', num2str(n_trainable_parameters))) end model.delete(); .. code-block:: matlab Loading and evaluating model-1 from checkpoint file. Number of trainable parameters for this temporary model:15 Loading and evaluating model-2 from checkpoint file. Number of trainable parameters for this temporary model:29 Loading and evaluating model-3 from checkpoint file. Number of trainable parameters for this temporary model:43 Loading and evaluating model-4 from checkpoint file. Number of trainable parameters for this temporary model:57 Loading and evaluating model-5 from checkpoint file. Number of trainable parameters for this temporary model:57 Loading and evaluating model-6 from checkpoint file. Number of trainable parameters for this temporary model:145 Loading and evaluating model-7 from checkpoint file. Number of trainable parameters for this temporary model:145 Loading and evaluating model-8 from checkpoint file. Number of trainable parameters for this temporary model:145 Loading and evaluating model-9 from checkpoint file. Number of trainable parameters for this temporary model:52 Now let's create a new Regressor object and load the model and extract information about it, such as the number of inputs, the number of outputs and the weights array: .. code-block:: matlab evaluator = py.NeurEco.NeurEcoTabular.Regressor(); load_status = double(evaluator.load('./EnergyConsumption/EnergyConsumption')); if load_status ~= 0 disp('Loading state = Fail') else % extracting general information disp('Loading state = Success') n_inputs = double(evaluator.get_input_count()); n_outputs = double(evaluator.get_output_count()); py_weights = evaluator.get_weights(); weights = double(py.array.array('d', py.numpy.nditer(py_weights)))'; disp(strcat('Number of Inputs :', num2str(n_inputs))); disp(strcat('Number of Outputs :', num2str(n_outputs))); disp(strcat('Number of trainable parameters :', num2str(size(weights, 1)))); .. code-block:: text Loading state = Success Number of Inputs :5 Number of Outputs :1 Number of trainable parameters :52 .. note:: When a NeurEco method returns a string, it can be converted by using the char() function, when it returns a numerical it can be converted using the double() function. For evaluation the following script can be used: .. code-block:: matlab n_test_samples = py.int(size(x_test, 1)); input_test = py.numpy.reshape(py.numpy.array(reshape(x_test.',1,[])), py.tuple({n_test_samples, py.int(n_inputs)})); output_test = py.numpy.reshape(py.numpy.array(reshape(y_test.',1,[])), py.tuple({n_test_samples, py.int(n_outputs)})); neureco_outputs_py = evaluator.evaluate(input_test); neureco_outputs = reshape(double(py.array.array('d', py.numpy.nditer(neureco_outputs_py))), size(x_test, 1), n_outputs); Testing data need to be converted to numpy array (like for the build), but the method evaluate will return a numpy array (neureco_outputs_py), so the numpy array need to be transformed into an nditer and then to MATLAB using the method double(). The shape of the output matrix will be lost at this point so it need to be reshaped. The L2 error can be computed to check how good the model is on the unseen testing data, example: .. code-block:: matlab l2_error = evaluator.compute_error(neureco_outputs_py, output_test); disp(strcat('L2 relative error (%) on testing set:', num2str(100 * l2_error))) .. code:: text L2 relative error (%) on testing set:8.567 The model can be saved to a different directory, under a different name: .. code-block:: matlab save_status = double(evaluator.save('EnergyConsumption/NewDir/SameModel')); if save_status == 0 disp('Saving state = Success') else disp('Saving state = Fail') end .. code:: text Saving state = Success The evaluator can be deleted then a new NeurEco Regressor can be created to export the model (this step is unnecessary, it is proposed for the sake of the code's clarity). The model can be exported to a C-file, an ONNX file, an FMU file or a bas file. .. code-block:: matlab evaluator.delete() % Create a model to load and export the NeurEco regressor exporter = py.NeurEco.NeurEcoTabular.Regressor(); load_status = double(exporter.load('./EnergyConsumption/EnergyConsumption')); if load_status ~= 0 disp('Loading state = Fail') else % export C Model disp('Exporting header File') exporter.export_c('./EnergyConsumption/EnergyConsumption.h', 'float'); % export ONNX Model disp('Exporting ONNX File') exporter.export_onnx('./EnergyConsumption/EnergyConsumption.onnx', 'float'); % export FMU Model disp('Exporting FMU File') exporter.export_fmu('./EnergyConsumption/EnergyConsumption.fmu'); % export VBA Model disp('Exporting VBA File') exporter.export_vba('./EnergyConsumption/EnergyConsumption.bas'); end exporter.delete()